The General Two Sample Problem

The two sample problem is concerned with data consisting of two mutually independent random samples. Not only are the elements within each sample independent, but also every element in the first sample is independent of every element in the second sample.
The universe consists of two populations, namely X and Y. Let the cumulative distribution function of X be called \(F_X\) and that of Y be called \(F_Y\). We draw a sample of size n from the X population and a sample of size m from the Y population.
Let these be denoted by \(X_1,....,X_n\) and \(Y_1,....,Y_m\) respectively. Usually the hypothesis of the two sample problem involves testing if the two samples are from identical populations i.e. \(H_0:F_X(x)=F_Y(x)\) for all x.
In practice, we test for if there is a location shift or a scaling factor between the two CDF’s.

Two Sample Location Problem

Suppose we draw two independent samples of sizes m and n from two continuous populations, so that we have N=n+m observations in total.
Here we want to test the null hypothesis that the samples are drawn from identical populations against the alternative that the populations are of the same form but with a different measure of central tendency i.e. there is a location shift.
Symbolically, this can be expressed as:
\(H_0: F_Y(x)=F_X(x)\) for all x against
\(H_1: F_Y(x)=F_X(x-\theta)\) for all x and some \(\theta \neq 0\)
Under \(H_1\), the CDF’s of the of the populations X and Y are of the same functional form, but the CDF of Y is shifted to the left if \(\theta<0\) and shifted to the right if \(\theta > 0\).
We say Y’s are stochastically larger than the X’s if \(\theta>0\) and the Y’s are stochastically smaller than the X’s when \(\theta<0\).

Empirical CDF plot
The above plot shows the empirical CDF’s of two independent random samples each of size 100 from the N(0,1) (say \(X_1,...,X_{100}\) ) and the N(2,1) distributions (say \(Y_1,...,Y_{100}\) ). This is an example of location shift where we can say the X’s are stochastically smaller than the Y’s.

Empirical CDF plot

 The above plot shows the empirical CDF’s of two independent random samples each of size 100 from the N(0,1) (say \(X_1,...,X_{100}\) ) and the N(-2,1) distributions (say \(Y_1,...,Y_{100}\) ). This is another example of location shift where we can say the X’s are stochastically larger than the Y’s.

Suppose we want to show draw two independent samples of sizes 600 and 400 respectively from Shifted Exponential with mean=1 and shift=2 and Exponential with mean=1.
This is an example of a two sample location problem. The random samples drawn from these distributions are plotted in the following graph: Sample Exponential Distribution plot

In the above graph, the location shift is clearly visible.

Parametric case:

If we have reason to believe that the samples are drawn from two independent normal populations, we can use the parametric test known as the 2-sample t test. Suppose \(X_1,...,X_n\) and \(Y_1,...,Y_m\) are independent samples drawn from \(N(\mu_X ,\sigma^2 )\) and \(N(\mu_Y , \sigma^2)\) respectively. (Here we assume that the samples are drawn from populations with equal variances.)
To test: \(H_0: \mu_Y-\mu_X = 0\) versus \(H_1: \mu_Y-\mu_X < 0\)
This testing is equivalent to testing the general alternative \(\theta=\mu_Y-\mu_X < 0\).

The best parametric test for testing this is the t-test with (m+n-2) degrees of freedom given by the following formula :
\[ t = \frac{\overline{X} - \overline{Y}}{\sqrt{s_u^2(\frac{1}{n} + \frac{1}{m})}} \ \sim t_{n+m-2} \] \[ where \ s_u^2 = \frac{(n-1)s_X^2 + (m-1)s_Y^2}{m+n-2} \] We reject \(H_0\) in favour \(H_1\) for large values of t. Suppose we consider the level of significance as \(\alpha=0.05\), then we reject \(H_0\) for \(t>t_{0.05 \ , \ m+n-2}\) where \(t_{0.05 \ , \ m+n-2}\) is the upper \(\alpha\) point of the \(t_{m+n-2}\) distribution.

Robustness of t-test:

The t-test is known to be more or less robust to non-normality as long as the distribution is more or less symmetric. However, when the distribution is asymmetric, the t-test does not perform well. In this project, we will compare the powers of the t-test with a non-parametric test, the Van der Waerden test for several distributions (both symmetric and asymmetric).

Van der Waerden test:

Let \(X_1, X_2,..., X_n\) be an iid random sample of size n from a distribution having cdf \(F\).
Let \(Y_1,Y_2,...,Y_m\) be an iid random sample of size m from a distribution having cdf \(G\).
Here \(F(x)=G(x- \theta)\).
\(\{X_i \}_{1\leq i\leq n}\) and \(\{Y_i \}_{1\leq i\leq m}\) are assumed to be independent samples.   Suppose we want to test the hypothesis \(H_0:\theta=0\) against \(H_1: \theta > 0\)
The Van der Waerden test statistic is given by:
\[V= \sum_{i=1}^{n} \Phi^{-1} (\frac{R_i}{N+1})\]

The statistic is a linear rank statistic as it can be written in the form of \(\sum_{i=1}^{n} a(R_i)c_i\) where \(a(i)= \Phi^{-1} (\frac{i}{N+1})\) and \(c_i= 1\) if i=1,2,…,n and 0, otherwise

Under \(H_0\)
\(E_{H_0}[V]=0\) and \(Var_{H_0}[V]= mn \frac{ \sum_{i=1}^{N} [\Phi^{-1} (\frac{i}{N+1})]^2}{N(N-1)}\)

Test: Reject \(H_0\) in favour of the alternative \(H_1 : \theta >0\) for large values of V.

Distribution Free Property of van der Waerden test:

Under \(H_0\), the Van der Waerden test statistic is distribution free. We will try to show this using simulations. We will test with samples drawn from N(0,1), Exp(1), U(-1,1), Beta(5,1).
We will consider the following sample sizes (n,m) :
(10, 10) (without standardizing)
(10, 10)
(20, 10)
(20, 20)

For each chosen sample size and distribution, we generated 1000 samples of \(X's\) and \(Y's\) under \(H_0\). The histogram and empirical CDF of the obtained Van-der waerden test statistic values is plotted here :

m=10 n=10 plot m=10 n=10 plot If we standardize the statistic we can also get a clue of the asymptotic behavior of Van-der waerden statistic. The following plots are for standardized statistic values.

m=10 n=10 plot

m=10 n=10 plot m=10 n=20 plot

m=10 n=20 plot m=20 n=20 plot m=20 n=20 plot

For all the histograms, we can see that under several different distributions the plots of the Van der Waerden statistic looks almost similar. This is further emphasized by the use of Empirical CDF plots, which are almost overlapping for the various distributions.

We also try using the two sample Kolmogorov Smirnov test on the test statistic obtained from N(0,1) and Exp(1) distributions.


In each case the p-value is way above \(\alpha = 0.05\) indicating the underlying distributions are same. Hence we can conclude, Van-der waerden statistic is distribution free under \(H_0\).

Asymptotic Distribution

For large sample size, the null distribution of the standardized Van der Waerden test statistic is well approximated by the standard normal distribution.
i.e. \[ \frac{V-E_{H_0}(V)}{\sqrt {V_{H_0}(V)}} \xrightarrow{d} N(0, 1) \] \[ where \ E_{H_0}(V)=0 \ and \ V_{H_0}(V) = mn \frac{ \sum_{i=1}^{N} [\Phi^{-1} (\frac{i}{N+1})]^2}{N(N-1)} \]

We will try to prove this using simulation, by the use of density plots and QQ-plots. We considered samples from \(Exp(1)\) distribution.
For the sample sizes, we have used the following method:
- Fixed n=10, m increasing
- Fixed m=10, n increasing
- Fixed n=5, m increasing
- Fixed m=5, n increasing
- Both n, m increasing

[For each of the following density plots, the dotted brown line indicates the density of the standard normal distribution.]

n=10, m increasing plot

m=10, n increasing plot

n=5, m increasing plot

m=5, n increasing plot

Both m, n increasing plot

Thus we see that for all the density plots, the density of the standardized Van der Waerden test statistic overlaps more or less with the density plot of the standardised normal distribution.
Also, from the Normal QQ-plot, we find that for small sample sizes, the tails of statistic deviate slightly from \(x=y\) line but as sample size increases the points are more or less on the \(x=y\) line, i.e. the sample quantiles of the standardized Van der Waerden test statistic corresponds more or less with the theoretical quantiles of a standard normal distribution.
This indicates that we can safely conclude that the standardized Van der Waerden test statistic is well approximated by the standard normal distribution for a large enough sample size.


Our next goal might be to understand how large the sample size should be so that the standardized Van der Waerden test statistic converges in distribution to the standard normal distribution.
For this, we fix an n in the range of 1 to 20, and take m from 5 to 50, incrementing by 5.
Our objective is to find for what sample size, the Shapiro Wilk test of normality yields a \(p-value \geq 0.05\). This is performed for 1000 replications and repeated 10 times. Each repetition is plotted below :

n,m normality
We see that as n increases, lower values of m is required to attain p-value of Shapiro Wilks test \(\geq 0.05\). In this plot, the different coloured lines represent the 10 replications. We see that when \(n = 1\), we need approximately \(m = 250\) to attain normality.
We ignore n=1, since then the graph will be easier to interpret.
n,m normality
From this, we observe that for \(n,m>15\), we can safely say that the standardised Van der Waerden test statistic converges in distribution to N(0,1) distribution.

Distribution under alternative:

Under \(H_0\), we consider \(\theta=0\).
We consider the following alternatives \(H_1:\theta=-1\) and \(H_2:\theta=1\).
We plot the density plots and Normal QQ plot of the standardized Van der Waerden test statistic under the null and the two alternative hypotheses.
alter dist
This is the density plot and QQ plot of the standardized Van der Waerden test statistic when we draw two independent samples of sizes n and m respectively from N(0,1) and N(\(\theta\),1) distributions.(Here we consider n=m=20)
alter dist
This is the density plot and QQ plot of the standardized Van der Waerden test statistic when we draw two independent samples of sizes n and m respectively from Exp(1) and Shifted Exponential with mean 1 and location shift \(\theta\) distributions.(Here we consider n=m=20)
The dotted red line indicates the density of the standard normal distribution.

We see that for \(\theta=-1\), the distribution of the standardized Van der Waerden test statistic is shifted to the left and for \(\theta=1\), the distribution is shifted to the right.
This indicates that if suppose the samples are drawn from Shifted Exponential with \(\theta=-1\), we have reason to reject the null hypothesis for small values of the Van der Waerden test statistic.
Similarly for \(\theta=1\), we have reason to reject for large values of the test statistic.
From the Normal QQ plot too, we observe a similar result.

Critical values:

We consider the following setup. We draw two independent samples of sizes n and m from distributions with CDF \(F_X\) and \(F_Y\) respectively, where \(F_X(x)=F_Y(x-\theta)\) for all x. To test
i> \(H_0: \theta=0\) vs \(H_1: \theta>0\)
ii> \(H_0: \theta=0\) vs \(H_2: \theta<0\)
iii> \(H_0: \theta=0\) vs \(H_2: \theta\neq0\)
(Here we have considered sample from Shifted Exponential Distribution with mean=1 and location shift \(\theta\). The distribution from which we draw the samples is immaterial as we have already shown that the test statistic is distribution free under \(H_0\).) We have considered sample sizes, m and n, from 10 to 50, incrementing by 10.
For the testing against \(H_1\), we reject the null hypothesis for large values of the Van der Waerden test statistic.
For the testing against \(H_2\), we reject the null hypothesis for small values of the Van der Waerden test statistic.
For the testing against \(H_3\), we reject the null hypothesis for both large and small values of the Van der Waerden test statistic.

The critical values were found by drawing 1000 samples of \(\{X's\}\) and \(\{Y's\}\) of size (n, m) under \(H_0\). - For the testing against \(H_1\), we calculated the upper \(\alpha\) quantile of statistic values.
- For the testing against \(H_2\), we calculated the lower \(\alpha\) quantile of statistic values.
- For the testing against \(H_3\), we calculated the upper \(\alpha/2\) quantile and lower \(\alpha/2\) quantile of the statistic values.  

Considering \(H_0\) vs \(H_1\):

This is a 3D plot representing the change in critical values with the change in sample size, when we reject for large values of the test statistic. We have the sample sizes on the X and Y axes and the critical values on the Z axis. We observe that as the sample size increases the critical value increases, when we reject the null hypothesis for large values of the test statistic.


Considering \(H_0\) vs \(H_2\):

The 3D plot is defined in the same way as above except here we reject for small values of the test statistic. We observe that as the sample size increases the critical value decreases, when we reject the null hypothesis for small values of the test statistic.

Considering \(H_0\) vs \(H_3\):

In this 3D plot, the upper plane represents the upper critical values and the lower plane represents the lower critical values. We observe that as sample size increases, the gap between the 2 planes increases, i.e. the lower critical decreases and the upper critical value increases with the increase in sample size.

We know that \(Var_{H_0}[V]= mn \frac{ \sum_{i=1}^{N} [\Phi^{-1} (\frac{i}{N+1})]^2}{N(N-1)}\) .
As sample size increases (i.e. as m and n increase), the variance of the test statistic under the null distribution increases. This in turn leads to increase in the upper critical value and decrease in the lower critical value since the dispersion of the test statistic under the null increases.
To verify this we have run a simulation.

Change in Variance

Power Comparisons:

We compare the powers for the t-test (parametric) vs the Van der Waerden (non-parametric) test for samples drawn from different distributions and different sample sizes.
We consider the following setup. We draw two independent samples of sizes n and m from distributions with CDF \(F_X\) and \(F_Y\) respectively, where \(F_X(x)=F_Y(x-\theta)\) for all x.
We test \(H_0: \theta=0\) vs \(H_1: \theta>0\)
We keep m and n equal and vary them from 10 to 50, incrementing both by 10 each time. All the tests are performed at level \(\alpha=0.05\)

For parametric counterpart :
We usually test for hypothesis: \(H_0: \mu_Y-\mu_X = 0\) versus \(H_1: \mu_Y-\mu_X < 0\)
Under the current setup the above hypothesis are equivalent to :
\(H_0: \theta=0\) vs \(H_1: \theta>0\) which are the concerned hypothesis.

We reject \(H_0\) in favour \(H_1\) for large values of t. Suppose we consider the level of significance as \(\alpha=0.05\), then we reject \(H_0\) for \(t>t_{0.05 \ , \ m+n-2}\), where \(t_{0.05 \ , \ m+n-2}\) is the upper \(\alpha\) point of the \(t_{m+n-2}\) distribution.

N(\(\theta\),1):
We draw a random sample of size n from \(N(\theta,1)\) distribution and a random sample of size m from N(0,1) distribution.
The power curves are as follows:

We observe that for small sample sizes (i.e for n=m=10) the t-test has greater power than the Van der Waerden test. [This is expected as for a normal distribution the t-test is the most powerful test for comparing means.] But we see that as the sample size increases, the gap between the power curves of the t-test and the Van der Waerden test goes on decreasing and becomes almost minimal for large enough sample sizes.
This is due to the fact that the Fisher Yates test is asymptotically optimum against the alternative that the populations are both normal distributions with same variance but different means. Under the classical assumptions for a test of location then, its ARE is 1 relative to Student’s t test. But we also know that asymptotically Fisher Yates and Van der Waerden tests are equivalent. So it makes sense that for larger sample sizes, the t-test and Van der Waerden test power curves are very close to each other.
For certain other families of continuous distributions, the Fisher Yates test is more efficient than the t-test (ARE>1) (Chernoff and Savage 1958). The examples of this are as follows:

Shifted Exponential with mean 1 and location shift \(\theta\)
We draw a random sample of size m from Exp(1) distribution and a random sample of size n from Shifted exponential distribution with mean 1 and location shift \(\theta\).
The power curves are as follows:

For the samples drawn from the shifted exponential, the Van der Waerden test performs consistently better than the t-test for all the sample sizes. [This is because t-test is not robust to skewness.]

U(\(-1+\theta,1+\theta\)):
We draw a random sample of size m from U(-1,1) distribution and a random sample of size n from U(\(-1+\theta,1+\theta\)).
The power curves are as follows:

For the samples drawn from the uniform distribution, the Van der Waerden test performs consistently better than the t-test for all the sample sizes.

Shifted Beta Distribution from \(\beta(5,1)\) by a parameter \(\theta\):
[Theory:
A beta random variable X, defined on the interval [0,1], can be rescaled and shifted to obtain a beta random variable on the interval [a,b] of the same shape by using the transformation Y=a+(b-a)X leading to \[f(Y)=\frac{1}{B[\alpha,\beta]}X\frac{(Y-a)^{\alpha-1}(b-Y)^{\beta-1}}{(b-a)^{\alpha+\beta-1}}\] where
a\(\le\)Y\(\le\)b
0<\(\alpha\)
0<\(\beta\)]
In this case, we consider a=\(\theta\), b=1+\(\theta\), \(\alpha\)=5, \(\beta\)=1. So the location shift is \(\theta\) and the scale factor is 1 (i.e. unscaled).
We draw a random sample of size m from Beta(5,1) distribution and a random sample of size n from Shifted Beta(5,1) distribution with location shift \(\theta\).
The power curves are as follows:

We see that here also the Van der Waerden test performs better than the t-test. This difference in performance is most notable for smaller sample sizes, and the difference reduces as sample size increases.
Also for Beta distribution we see that both the test attain power 1 very quickly (i.e. for small values of location shift).

Shifted Logistic Distribution:
*The pdf of the shifted logistic distribution is given by:

\(f(x ; \theta)=\frac{e^{x-\theta}}{1+e^{(x-\theta)^2}}\) where \(\theta\) is the location shift.* We draw a random sample of size m from a logistic distribution and a random sample of size n from a logistic distribution with location parameter \(\theta\).
The power curves are as follows:

Here we see that the Van der Waerden test performs better than the t-test for all choices of m and n. However, again as m and n increases, the differences between the performances of the 2 tests decreases and becomes almost negligible.

Cauchy(\(\theta\),1) distribution:
We draw a random sample of size m from C(0,1) distribution and a random sample of size n from C(\(\theta\),1) distribution.
The power curves are as follows:

We observe that the Van der Waerden test performs way better than the t-test in this case. As the sample size increases, we see that the difference in performance between the 2 tests increases. Also, the t-test performs especially badly in case of Cauchy distribution.
This can be attributed to the following reason:
When we draw samples from the Cauchy distribution and find the distribution of the test statistic of the t-test, we observe that instead of having a unimodal distribution (like what is expected from the test statistic of the t-test to follow a t distribution, at least approximately), we get a bimodal density plot.
This is represented in the following plot:

This points to some gross violation of assumption of the t-test and hence it performs poorly.

Consistency of Van der Waerden test

Consistent tests:
A test is consistent for a specified alternative if the power of the test, when that alternative is true, approaches 1 as the sample size approaches infinity.
Our objective is to test if the Van der Waerden test is consistent.
We plot the power curve of the Van der Waerden test for different values of n and m. We take n=m from 10 to 50, incrementing by 10.
Here the test is such that we reject for large values of the test statisitc.


We observe that as m and n increases, the power curve becomes steeper ,i.e, the power of the test increases at every point of the alternative.
This may hint that as we keep on increasing the sample size, the class of alternatives for which the test attains power 1, keeps on increasing.
This may be reason enough to suspect that as m and n goes to infinity, the test will attain power 1, for all values of the alternative. Hence, the test is consistent.

Violation of assumption of continuity:

The assumption that we use while performing the Van der Waerden test is that the samples are drawn from a continuous distribution. Violation of this assumption will cause ties in ranks as there is possibility of equality of realised values of samples for a discrete distribution. This breaks down the theory of linear rank statistic testing.
We tried to show the possibility of tied ranks while drawing samples from a Binomial distribution and its impact on the power of the Van der Waerden test.

We observe that there are jumps in the power curve for the Binomial case, and it is not smooth like in the Normal case.

As we observe the distribution of Van-der waerden statistic in this case has different forms at \(\theta = 0\) , \(0 < \theta \leq 1\) , \(1 < \theta \leq 2\). This explains the jumps in the power curve.

Comparison between Mann Whitney and Van der Waerden test:

Mann-Whitney U statistic is defined as follows : \[ U = \sum_{i=1}^n \sum_{j=1}^m \phi(X_i, Y_j) \ where \ \phi(X_i, Y_j) = 1\{X_i > Y_j \} \] In the setup defined above for Van-der-waerden test, to test \(H_0 : \theta= 0\) against \(H_1 : \theta > 0\).
When \(\theta > 0\); X is stochastically greater than Y, hence we should reject U for large values of the statistic.

We will now compare the powers of Van der Waerden test and Mann Whitney test for various sample sizes and different distributions. All the tests are performed at level \(\alpha\)=0.05

N(\(\theta\),1):
We draw a random sample of size n from N(\(\theta\),1) distribution and a random sample of size m from N(0,1) distribution.
We observe the cases when m=n=40 and m=n=50.

We see that in both the cases the Van der Waerden test performs marginally better than the Mann Whitney test.
We know when the samples are drawn from Normal distribution, the Van der Waerden test is asymptotically the Locally Most Powerful Rank test. Hence we will try to show that for a small interval around 0, the Van der Waerden test has more power than the Mann Whitney test for a fixed level \(\alpha\) for a large enough sample size.
This is verified by our power plots (for samples sizes m=n= 40 and 50).

Shifted Exponential with mean 1 and location shift \(\theta\)
We draw a random sample of size m from Exp(1) distribution and a random sample of size n from Shifted exponential distribution with mean 1 and location shift \(\theta\).
We observe the cases when m=n=40 and m=n=50.

Here also we observe that the Van der Waerden tests perform better than the Mann Whitney test for sample sizes m=n=40 and 50.

U(\(-1+\theta,1+\theta\)):
We draw a random sample of size m from U(-1,1) distribution and a random sample of size n from U(\(-1+\theta,1+\theta\)).

Again the Van der Waerden test performs better than the Mann Whitney test for sample sizes m=n=40 and 50.

Shifted Beta Distribution from \(\beta(5,1)\) by a parameter \(\theta\):
We draw a random sample of size m from Beta(5,1) distribution and a random sample of size n from Shifted Beta(5,1) distribution with location shift \(\theta\).

For the beta distribution, the power curves of the two tests are almost overlapping. But if we look at the smaller interval of \(\theta\) in (0, 0.25), we can see that the Van der Waerden test performs better than the Mann Whitney test for sample sizes m=n=40 and 50.

Shifted Logistic Distribution:
We draw a random sample of size m from a logistic distribution and a random sample of size n from a logistic distribution with location parameter \(\theta\).

When the samples are drawn from Logistic distribution, the Mann Whitney test is Locally Most Powerful Rank test.
When we observe a small interval around \(\theta\)=0, we see that the Mann Whitney test performs marginally better than the Van der Waerden test.


Cauchy(\(\theta\),1) distribution:
We draw a random sample of size m from C(0,1) distribution and a random sample of size n from C(\(\theta\),1) distribution.

Hence the Mann Whitney test performs better than the Van der Waerden test when we take sample sizes m=n=30 and 40.
[Note: All the tests here have lowest power at \(\theta =0\) (or in some cases some value of \(\theta\) very close to 0, which maybe due to the randomness due to simulations) and for every other \(\theta\) in alternative the power is greater that \(\alpha\). Hence, we can say that both the tests are unbiased.]

Unequal Sample Sizes:

We will now compare t-test, Van der Waerden test and and Mann Whitney test for unequal sample sizes (i.e. unequal values of m and n). All the tests are again performed at level \(\alpha=0.05\).

N(\(\theta\),1):
We draw a random sample of size n from \(N(\theta,1)\) distribution and a random sample of size m from N(0,1) distribution.
We take (n.m)=(20,50) and 10,40)

We observe that all the tests have almost the same power for unequal sample sizes.
But for a small region around \(\theta\) we observe that the Van der Waerden test has more power than the Mann Whitney test, which is expected since the Van der Waerden test is asymptotically the LMP rank test.

Shifted Exponential with mean 1 and location shift \(\theta\)
We draw a random sample of size m from Exp(1) distribution and a random sample of size n from Shifted exponential distribution with mean 1 and location shift \(\theta\).
We take (m,n)=(5,50) and (20,50)

For sample size (5,50), we see that the Mann Whitney test performs better in most regions than the Van der Waerden test.(This is kind of in contradiction to what we observed when m=n=40 and 50. But since the powers of the two tests are quite close we may conclude that this is due to the randomness of simulations). But both the non-parametric tests perform way better than the t-test.
For sample size (20,50), we see that the non parametric tests have almost equal power, but again they perform way better than the t-test.

U(\(-1+\theta,1+\theta\)):
We draw a random sample of size m from U(-1,1) distribution and a random sample of size n from U(\(-1+\theta,1+\theta\)).
We take (m,n)=(5,50) and (20,50).

When we take sample size (20,50), all the three tests have almost equal powers. But when we take sample size (5,50), we see that the powers of the t-test and the Van der Waerden test are almost same. But the Mann Whitney test performs comparitively poorly.

Shifted Beta Distribution from \(\beta(5,1)\) by a parameter \(\theta\):
We draw a random sample of size m from Beta(5,1) distribution and a random sample of size n from Shifted Beta(5,1) distribution with location shift \(\theta\).
We take (m,n)=(5,50) and (20,50).

For samples drawn from Beta distribution, all the tests attain power=1 very quickly. So we have also looked at a small interval of \(\theta\) to draw a more meaningful conclusion from the plots.
For both sample sizes (5,50) and (20,50), Van der Waerden test performs better than all the other tests.


Shifted Logistic Distribution:
We draw a random sample of size m from a logistic distribution and a random sample of size n from a logistic distribution with location parameter \(\theta\).
We take sample sizes (20,50) and (10,20).


For both the sample sizes, we can observe that the Mann Whitney test performs best. This is expected as Mann Whitney is LMP Rank test when the underlying distribution is Logistic.

Cauchy(\(\theta\),1) distribution:
We draw a random sample of size m from C(0,1) distribution and a random sample of size n from C(\(\theta\),1) distribution.
We take (m,n)=(5,50) and (20,50).


For both the sample sizes, Mann Whitney performs best. However, the more significant observation here is that the t-test performs significantly poorly when the samples are drawn from a Cauchy distribution.

Conclusion:

Acknowledgement and Bibliography

We would like to thank Prof. Isha Dewan for her continuous guidance in our project and dedication in teaching. We have also taken help of the book Nonparametric Statistical Inference by Gibbons and Chakraborti for theory that we have used in the project.